Add socket manifest dynamic-sbom-inference - #1467
Add socket manifest dynamic-sbom-inference#1467Jeppe Fredsgaard Blaabjerg (jfblaa) wants to merge 30 commits into
Conversation
Recursively discovers independent gradle/sbt/maven build roots and generates a Socket facts SBOM for each, without re-invoking the manifest script on subproject/reactor-module directories a parent build root already covers. Maven reactors in particular have a pom.xml per module, so naive marker-file discovery would over-invoke; coverage is tracked per ecosystem using the facts SBOM's own projects[].subprojectDir, which every build-tool producer already reports. This is a standalone command rather than an --auto-manifest extension, so it can force stricter behavior later without touching auto's existing contract. Scoped narrowly for this first PR: discovery plus generation only, one global socket.json config applied to every discovered root (no per-build-root cascade), no new CLI flags beyond --exclude-paths/--verbose. Refs REA-685, REA-553
--exclude-paths on socket manifest auto/gradle/kotlin/maven/scala/ dynamic-sbom-inference reused the flag description from the scan/reach flag definitions verbatim, which talks about "the scan" and full application reachability analysis. None of these commands scan or run reachability analysis themselves, so the wording was confusing when viewed via --help on any of them standalone. Adds a manifest-scoped excludePathsFlag with wording specific to manifest/facts generation and switches all six commands to it.
…ation
Different projects in a repo may need different JDKs for their build
tool. Adds a javaHome field to defaults.manifest.{gradle,maven,sbt} in
socket.json, configurable via the socket manifest setup wizard, and
threads it through to the actual build-tool invocation (overrides
JAVA_HOME for that spawn only, everything else about the environment is
left untouched).
Applies to socket manifest gradle/kotlin/maven/scala, socket manifest
auto, and socket manifest dynamic-sbom-inference. Scoped to the Socket
facts generation path only, not the legacy --pom conversion path. No
new CLI flag - socket.json/the setup wizard is the only configuration
surface for now.
socket manifest dynamic-sbom-inference previously read socket.json once
at the overall recursion root and applied that single config to every
discovered build root. It now resolves each build root's own nearest
socket.json (walking up from that root, bounded at the recursion root,
nearest wins - no merging), so different projects in the same repo can
carry their own settings instead of being forced onto one shared config.
Also adds $VAR/${VAR} expansion for javaHome, resolved against the CLI
process's own environment. A hardcoded absolute JDK path only works on
whoever's machine wrote it; referencing an env var each developer sets
themselves (e.g. $JAVA11_HOME) makes a shared socket.json portable
across machines. Fails closed with a clear message if the referenced
variable isn't set, rather than silently passing a broken path to the
build tool.
readSocketJsonCascade (renamed from readOrDefaultSocketJsonUpTo) now merges defaults.manifest.<ecosystem> field-by-field across every ancestor between a build root and the recursion root, nearest winning per field, instead of one file replacing the root config wholesale. A subproject can now override just javaHome while still inheriting the root's excludeConfigs/bin/etc., rather than having to restate the whole config. Ecosystems the override doesn't mention are left untouched. Verified against the sandbox tree: a root socket.json setting excludeConfigs plus a nested one setting only javaHome both applied together for that build root.
dynamic-sbom-inference always generates Socket facts SBOMs, never pom.xml. If a build root's cascaded socket.json sets facts: false (the pom-mode opt-out other manifest commands honor), that's ignored here rather than skipping the project or silently doing nothing - but it's a real, deliberate setting the user made, so it's surfaced as a warning rather than silently overridden.
dynamic-sbom-inference previously forced Socket facts generation over an explicit defaults.manifest.<ecosystem>.facts: false (pom mode), just warning about it. Reverted: facts: false now skips the project again, same as before that change, since this command has no pom-mode equivalent to fall back to. Also adds a separate ignored: true field (gradle/maven/sbt) scoped specifically to dynamic-sbom-inference, for projects that should be skipped during recursive generation regardless of their facts/pom preference for other commands. Kept as its own boolean rather than folding into facts, since facts already means something specific to other manifest commands and overloading it would require touching every consumer for a value only this command understands. New skippedIgnored outcome status covers both reasons, distinguished via the warning message.
A cascaded (not just root-level) disabled: true produces identical per-root behavior to the ignored field just added, so keeping both was redundant. Removed ignored; disabled now does double duty: root-only ecosystem-wide gating for auto/gradle/etc. (unchanged), plus a cascaded per-build-root skip specifically for dynamic-sbom-inference. Renamed the skippedIgnored outcome status to skippedDisabled to match.
…ursive setup for dynamic-sbom-inference - socket.json manifest fields now accept `null` as an explicit "clear the inherited value" sentinel, distinct from leaving a field unset; the setup wizard writes it when a previously-set value is cleared instead of just deleting the key. - Add a lightweight per-ecosystem workspace enumeration path for gradle/sbt/maven (new standalone scripts, kept fully separate from the existing facts-generation scripts) that discovers a build's subprojects without running dependency resolution. - Add a hidden `socket manifest setup --dynamic-sbom-inference` mode: configures root-level defaults per ecosystem, then recursively marks `disabled: true` on build roots matching `--exclude-paths`, leaving everything else untouched. - Speed up `dynamic-sbom-inference`'s handling of a disabled build root with many nested candidates by reusing the nearest already-resolved disabled ancestor instead of re-walking the whole config cascade for each one, and only logging the root cause instead of once per nested candidate.
Previously each excluded build root got its own disabled:true write, relying on cascade to skip descendants already covered by an ancestor write. That only worked when the excluded ancestor happened to be a build root itself; a non-project directory containing multiple sibling projects would leave later siblings enabled. Instead, group excluded projects by the shallowest directory that actually matches --exclude-paths and write disabled:true there once, covering every ecosystem and sibling/nested project beneath it regardless of whether that directory is a build root of its own.
… is unknown A build root whose facts generation fails (a build-tool crash or a blocking resolution failure) never produces its projects[] list, so there's no way to tell whether a later candidate underneath it is already covered by that root or a genuinely independent project. Continuing to process further candidates in that state risked misclassifying subprojects and piling on doomed attempts against a build already known to be broken. Fail closed instead: abort the entire recursive walk as soon as one build root's workspace layout can't be determined, rather than continuing to sibling and nested candidates.
The recursive wizard (`socket manifest setup --dynamic-sbom-inference`) previously only ever disabled build roots matching --exclude-paths; every other discovered root was left completely untouched, with no way to set its bin/JDK/opts short of running the plain single-project wizard on it directly. Every non-excluded candidate now gets an interactive configure-or-inherit-defaults prompt (in discovery order, parent before child), seeded with its cascaded effective value so accepting every prompt unchanged preserves whatever it already inherits. Disabling a specific candidate is intentionally not offered here - that stays --exclude-paths' job, so a whole excluded subtree still collapses into a single write. Also fixes a real bug surfaced along the way: askForBin pre-filled the hardcoded tool fallback (mvn/./gradlew/sbt) as the prompt's shown value, so accepting the default was indistinguishable from explicitly typing it and got written to socket.json for no reason. The shown default is now only ever a prior explicit value; the fallback is mentioned as a hint instead. A matching guard drops any ecosystem section that ends up empty so it doesn't count as configured or trigger a write. Finally, the root step now detects which ecosystems are actually present at cwd (reusing the same check the plain wizard uses) and asks about detected ecosystems first, phrased accordingly, before offering to configure undetected ones "anyway" for subprojects that might need them.
Fixes two correctness bugs found in an audit pass: leaving a config prompt blank when the field was already explicitly cleared (null) deleted the key instead of preserving the clear, silently reverting it to inheriting an ancestor's value; and --exclude-paths never reached the wizard's workspace enumeration, so excluding a broken reactor member didn't stop the wizard from still trying to resolve it and aborting the whole walk. Also tightens several UX rough edges in the recursive wizard: drops a redundant write confirmation inconsistent with the rest of the flow, fixes a tally line that double-counted re-enabled candidates and never reported disabled ones, and softens wording that overclaimed knowledge the wizard doesn't actually have yet (a build root at cwd, a fixed --exclude-paths prompt count). Trims the surrounding comments down to non-obvious rationale only, per repo comment-style guidelines.
Cuts several multi-line comment blocks down to their non-obvious why, matching the repo's comment-style guidelines.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 5 potential issues.
Bugbot Autofix is ON. A cloud agent has been kicked off to fix the reported issues.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 36f354f. Configure here.
- A root-disabled ecosystem was dropped from the build-tool scan entirely, so a nested socket.json could never re-enable it. Moved the strip-before- scan helper (previously wizard-only) to discover-manifest-roots.mts and applied it in generateRecursiveManifests too, letting the existing per-directory cascade check be the sole arbiter of skip vs. include. - runManifestFacts's progress line used logger.log (stdout), polluting --json output with non-JSON lines ahead of the payload. Switched to logger.info (stderr), matching every other status line in that function. - `socket manifest setup --dynamic-sbom-inference` forwarded --exclude-paths without validation, unlike every sibling manifest command. Added the same assertValidExcludePaths call. - Recursive generation inferred a build-root failure from whether process.exitCode changed during a call, which misclassifies a real failure as empty if the exit code was already non-zero beforehand. Gave runManifestFacts an explicit null (failure) vs. undefined (empty) return so callers don't have to infer it from global state. - Verbose error logging in enumerate-workspaces/run-manifest-facts string-coerced the caught error directly; switched to the existing getErrorMessageOr helper.
…cket-manifest-dynamic-sbom-inference-for-recursive
The merge brought package.json's version from 1.1.150-prerelease to a clean release version, so these inline snapshots no longer matched.
1.1.152 never actually published (still 404s on the registry, staged dist-tag is still 1.1.151), so bump to the next-version-hint convention used elsewhere in this repo's release history and refresh the CLI banner snapshots to match.
Drop reactor-member "skippedCovered" lines from the per-line table - they're implied by their parent's line already showing up, and the aggregate count in the summary still reports them. Also drop the "across N build root(s)" total from the summary line: it counted every candidate directory visited, including reactor members that aren't independent build roots, which overstated how many actually exist.
Drop failed/skipped/empty from the tally: a failure aborts the whole walk immediately rather than accumulating (and is already reported via its own fail message), and the disabled/covered/empty buckets count candidate directories rather than independent build roots, so a total there is just as misleading as the "N build root(s)" figure already removed.
Root cause of the flaky snapshot mismatches on this PR: whether the CLI itself redacts its version banner (VITEST baked in at build time) or prints the real one and leaves redaction to this test helper depends on env propagation into the build step, not just the test run - so the same source can produce either "<redacted>" or the real "vX.Y.Z-prerelease" depending on how it was built. normalizeBanner's regex only stripped a bare "vX.Y.Z", leaving a trailing prerelease suffix dangling in one case but not the other. Broadened it to match a trailing prerelease/build suffix and to be idempotent on an already-redacted value, so both cases normalize identically. Refreshed the now-correct snapshots.
Martin Torp (mtorp)
left a comment
There was a problem hiding this comment.
Nice work — the hard parts here (reactor coverage classification off projects[].subprojectDir, the field-level cascade, the explicit null-clear sentinel) are well reasoned, and the commit history shows real self-correction along the way. The comments explain why rather than what, which made this a pleasant read.
Approving, with two things I'd like fixed before this goes out — both small and localized, left as inline comments:
generateRecursiveManifestspairs realpath-resolved candidate dirs with a rawcwd, which silently breaks the cascade boundary and drops--exclude-paths.setupRecursiveManifestConfigalready guards against exactly this; the generation path just didn't get the same treatment.enumerateWorkspacesaccepts "exit 0, zero projects" as success, which turns a stale Maven extension jar into silent degradation rather than an error.
Non-blocking observations, for whenever you get to them:
break ecosystemsis broader than its rationale.coveredis allocated per-ecosystem, so a gradle failure carries no information about maven coverage classification — yet one failure stops every ecosystem, and the output can't tell the user that maven/sbt were never attempted. Breaking only the inner loop plus anabortedoutcome would say so.- The name is already taken.
scan create/scan reachhave carried a hidden--dynamic-sbom-inferencesince #1451, which passes--maven-use-only-root-socket-facts(use only root facts) and implies--auto-manifest. This command generates per-root facts recursively, via a code path the scan flow never invokes. Two unrelated features under one name, one of them now user-visible — worth settling before it's unhidden. - The new JVM producers have no automated coverage.
socket-workspaces.init.gradle,socket-workspaces.plugin.scala, andSocketWorkspacesRecordsEnginearen't referenced byscripts/test/run-compat.sh, which is the only thing exercising the bundled producers — and per its own README there's no CI for it. The files carry explicit compatibility claims (Gradle 1.0+, Scala 2.10/sbt 0.13 through 2.12/sbt 1.x) that nothing verifies, and sinceWORKSPACES_TASKis the sole entry point, a registration or compile error surfaces only as the silent degradation in comment 2. configureCandidatematerializes the inherited section into the child (pinned bysetup-recursive-manifest-config.test.mts:441—bin: './gradlew'comes only from the cascade and still gets written). So once a user picks "Configure", later edits to the parent'sjavaHome/bin/excludeConfigsstop reaching that candidate. Deliberate and tested, but it converts inheritance into a snapshot, which is surprising when a field-level cascade is the headline feature — worth either writing only genuinely-changed fields or saying so in the comment.--json --verboseemits invalid JSON.cmd-manifest-dynamic-sbom-inference.mts:92-96andrun-manifest-facts.mts:166,175uselogger.log(stdout) ahead of the payload. This is the only manifest command carryingoutputFlags, so the combination is newly reachable — and 58fa321 fixed exactly this one line away.- Smaller stuff: two stale
socket manifest setup --recursivereferences (enumerate-workspaces.mts:15,SocketWorkspacesRecordsEngine.java:17— the flag is--dynamic-sbom-inference);expandEnvVarRefshas no$$escape and reports only the first missing variable;renderTableemits a bare blank line when every outcome isskippedCovered; the exclude-glob variant expansion now lives in six producers, each commented as mirroring the others, so a shared fixture-driven test would keep them honest; the generation command is visible while its only config surface is hidden.
The e2e-tests failures on Node 20/22/24 are socket fix on a Python/django fixture, unrelated to this PR — type check, lint, and the unit matrix are green.
generateRecursiveManifests read candidate dirs realpath-resolved (findBuildToolCandidates already does this) but passed the raw cwd as the socket.json cascade boundary and as the anchor for re-anchoring --exclude-paths. Whenever cwd contains a symlink (macOS /tmp -> /private/tmp, several CI layouts), the boundary comparison never matched: the cascade walked all the way to the filesystem root instead of stopping at cwd, and --exclude-paths silently stopped reaching the build tool invocation. Resolve cwd once and use it consistently for both. Also realpath-resolve a project's subprojectDir before adding it to the covered set, in both generate-recursive-manifests.mts and setup-recursive-manifest-config.mts, so a symlinked reactor member is correctly recognized as covered instead of escaping and being reinvoked as an independent root. enumerateWorkspaces treated a clean exit with zero projects as success, but every real enumeration reports at least the build's own root project, so zero projects always means the task never ran (e.g. an extension jar built before the workspace-enumeration participant existed). Drop the exit-code condition so this is always a failure.
- Hide `socket manifest dynamic-sbom-inference` - its name collides with the unrelated, root-only --dynamic-sbom-inference flag on scan create/reach (different semantics: this one is recursive per-root). Keep it internal until that naming collision is resolved. - --json --verbose emitted invalid JSON: the verbose debug preamble and run-manifest-facts' verbose resolution-detail logging both wrote to stdout ahead of the JSON payload. This is the only manifest command with --json, so the combination was newly reachable. Gated the preamble on !json and switched the detail logging to stderr. - Fixed two stale "socket manifest setup --recursive" comments left over from an earlier flag name. - renderTable no longer prints a bare blank line when there are no non-covered outcomes to show.
…the cascade Every field shown to the sub-wizard (including ones inherited from an ancestor, never touched by the user) was written verbatim into the candidate's own file, permanently pinning that value against future changes to the ancestor - converting inheritance into a one-time snapshot despite the field-level cascade being the point. Now diffs the final seed against an ancestor-only baseline (the cascade computed from dir's parent, excluding dir's own file) and only writes fields that actually differ, so an untouched field keeps inheriting live.
socket-workspaces.init.gradle, socket-workspaces.plugin.scala, and CoanaWorkspacesLifecycleParticipant/SocketWorkspacesRecordsEngine had no automated coverage at all - the local compat matrix only exercised the facts scripts, so a registration or compile error in the workspaces siblings would surface only as silent degradation (the setup wizard's reactor-coverage pruning treating a real reactor as if it had no members). Adds a smoke-test-workspaces.sh per ecosystem, wired into the same per-version matrix run-compat.sh already runs for the facts scripts, asserting each variant emits exactly a meta record plus the expected project record(s) and nothing else (no node/root/file records, confirming no dependency resolution happens). Verified locally against the currently installed gradle/maven/sbt.
|
Thanks for the thorough review! Addressed the non-blocking observations:
Holding off on Left |
Coverage (`covered`/`disabledRoots`) is tracked per ecosystem, so a gradle failure carries no information about maven or sbt's own classification - yet aborting all remaining ecosystems meant an unrelated one could be blocked from ever starting, with nothing in the output explaining why. Narrowed the abort to the failing ecosystem's own loop, and added an 'aborted' status for that ecosystem's still-untried candidates so they show up explicitly instead of being silently absent from the output.
Only the first missing variable was ever reported, so a value referencing two unset vars needed two runs to discover both. Also had no way to represent a literal $WORD - $$ now expands to a literal $, so whatever follows it is left untouched. Added a dedicated test file; this function had none.
|
Follow-up on the two items from my earlier comment:
Left the shared fixture-driven test for the exclude-glob expansion (duplicated across the 6 producers) deferred — didn't want to expand scope further on this PR, but happy to take it on as a follow-up if you'd like. |
The six JVM producers (gradle/maven/sbt × facts/workspaces) each carried their own NIO PathMatcher-based exclude-glob implementation, including a "zero-depth variant expansion" workaround for a semantics mismatch between NIO's `**` and the CLI's micromatch. Following the same pattern already used for --include-configs/--exclude-configs (PR #1404), the glob is now compiled to a portable regex pattern source once in exclude-paths-glob.mts, transported via the existing -D/-P property mechanism, and each producer just Pattern.compile()s what it receives. Gradle/sbt/maven all share java.util.regex, so one dialect covers every producer. Verified end-to-end against real gradle, maven, and sbt invocations (facts and workspaces variants) with an actual --exclude-paths value.
|
Followed up on the "exclude-glob variant expansion now lives in six producers" observation: centralized Gradle/maven/sbt all run on the JVM and share |
…bom-inference-for-recursive

Summary
socket manifest dynamic-sbom-inference: recursively discovers gradle/sbt/maven build roots and generates a facts SBOM per independent root, with reactor/multi-module coverage tracking so submodules aren't re-invoked.socket manifest setup --dynamic-sbom-inferencewizard to scaffold per-projectsocket.jsonconfig across a tree, with a cascadingdefaults.manifest.*inheritance model and an explicitnullsentinel for clearing an inherited value.Test plan
pnpm check:tsc/pnpm check:lintpnpm build:dist:srcNote
Medium Risk
Large new surface around spawning Gradle/Maven/sbt and monorepo classification; mis-skips or fail-closed aborts could block SBOM generation on complex trees, but changes are localized to manifest CLI and bundled build-tool scripts.
Overview
Adds
socket manifest dynamic-sbom-inference, which walks a tree under CWD, finds Gradle/Maven/sbt build roots, and writes.socket.facts.jsonper independent root. Reactor members already covered by a parent’s factssubprojectDirlist are skipped; a failed root aborts the walk so later roots aren’t misclassified.socket manifest setupgains hidden--dynamic-sbom-inference(and--exclude-paths) to scan the tree, bulk-disable excluded paths, and interactively configure per-rootsocket.jsonwith cascade inheritance and explicitnullto clear inherited values.Lightweight workspace enumeration (Gradle/sbt/Maven init scripts/plugins/extension) lists projects without resolving dependencies, used by the setup wizard’s coverage pruning. Dist build copies
socket-workspacesassets alongside facts scripts.Manifest commands now use a shared
exclude-pathsflag description,javaHomefromsocket.json(with$VAR/${VAR}expansion intoJAVA_HOME), andrunManifestFactsreturns generated projects for recursive coverage.Reviewed by Cursor Bugbot for commit 36f354f. Configure here.